# DPO Fine-tuning

This is a 70B-parameter large language model based on Llama 3.3, fine-tuned with SFT+DPO, specializing in character-driven dialogues and creative content generation

An enhanced version of the Qwen2.5-14B model fused using the TIES method, focusing on instruction following and dialogue optimization

Tanuki-8B is an 8B-parameter Japanese large language model optimized for dialogue tasks through SFT and DPO, developed by GENIAC Matsuo Lab

Based on the Mistral 7B large language model, fine-tuned through pre-trained model merging on the Date-DPO-v2 dataset, excelling in role-playing, creative writing, and general intelligent tasks.

StarChat2 is a 16-billion-parameter programming assistant model fine-tuned based on StarCoder2, excelling in conversation and code generation tasks

Zephyr 7B Gemma is a language model fine-tuned based on google/gemma-7b, trained on publicly available synthetic datasets using Direct Preference Optimization (DPO), designed to serve as a helpful assistant.

Korean instruction-optimized model based on EEVE-Korean-Instruct-10.8B-v1.0, trained using Direct Preference Optimization (DPO) method

Minueza-32M-Chat is a chat model with 32 million parameters, based on Felladrin/Minueza-32M-Base and trained with supervised fine-tuning (SFT) and direct preference optimization (DPO).

OLMo 7B Instruct is an open language model trained on the Dolma dataset, optimized with SFT and DPO, specifically designed for QA tasks.

A 14B-parameter causal language model optimized with DPO, supporting English-Chinese text generation tasks

A 14B-parameter causal language model fully compatible with Meta LLaMA 2 architecture, outperforming all sub-70B models in multiple benchmarks